1 link tagged with all of: inference + verbalized sampling + diversity + llm alignment + mode collapse
Links
The paper shows that typicality bias in human preference data drives mode collapse in aligned LLMs, narrowing their output diversity. It introduces Verbalized Sampling, a prompt that asks the model to list multiple responses with their probabilities, and demonstrates significant diversity gains across writing, dialogue, QA, and synthetic data tasks without retraining.
- Mode collapse comes from "typicality bias" in human preference data—annotators favor typical/safe responses, so even a perfect reward model trained on that data would still collapse diversity.
- Verbalized Sampling (prompting the model to list multiple responses with probabilities) recovers pre-training diversity at inference time, no retraining needed.
- This boosts creative writing diversity 1.6–2.1x over direct prompting, with human judges rating outputs 25.7% better on average, and it also improves downstream math solver accuracy when used to generate training data—all without hurting factual accuracy or safety.
verbalized sampling
mode collapse
diversity
llm alignment
inference